Skip to content

fix(agc): put a stencil replay shader's guest buffer offsets where its shader data is - #8615

Merged
oneandonlydean merged 2 commits into
boykopovar:mainfrom
Lindamea1:fix/agc-stencil-replay-push-data
Oct 26, 2026
Merged

oneandonlydean merged 2 commits into
boykopovar:mainfrom
Lindamea1:fix/agc-stencil-replay-push-data

Conversation

@Lindamea1

@Lindamea1 Lindamea1 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What

On a device without VK_EXT_shader_stencil_export, a pixel shader that exports stencil is drawn by replay, which passes the reference in a push constant word after the shader data. When the shader data is not in push constants, that word is the whole push block (stencilReferenceDword 0): BindingAllocator::Allocate gives a shader data buffer to any shader with an image under registration modules, which follow the pipeline library, and to shader data larger than the stage push range. ShaderResources::buildPrepare took any non-empty push block to mean the guest buffer offsets are in push constants, so such a shader that reads a guest buffer fails with AGC graphics: guest buffer offset lies outside the shader's push constants (byte 64 against a 4-byte block in the first commit's test case).

Found while bringing up Monster Hunter Wilds (PPSA24191, #7972) on NVIDIA. On main 1e20650e8 with #10304 and the open stencil replay changes (#9317 with #9904, #9393, #9905 and #9906), the title's first replayed draw of an unbounded stencil export throws this error. With this PR, the run gets past it and stops at a later stencil replay check (see Tested). @TheSovietWaffle found the same failure in Onimusha: Way of the Sword (PPSA27836, #5769) on an RTX 3080, running it with a local stack of the open stencil replay changes, and opened #9427 with the same fix; its test case is the second commit here, with them as its author, as they agreed there. @akkadbakkad89 hit it in Assassin's Creed Shadows (PPSA20397, #8674) on an RTX 3070, in a local stack with #9317, #9393, #9904, #9905 and #9906 (see Tested). No title is known to reach this on main alone. The RTX 5090 used here lacks VK_EXT_shader_stencil_export and has the pipeline library, so on main every replayed stencil export there whose shader uses an image and reads a guest buffer throws.

The shader reads the offsets from push constants only when its layout puts the shader data there, and from the shader data buffer otherwise (EmitShaderDataDwordLoad). For a shader with stencil references, buildPrepare now counts only the stencilReferenceDword words before the reference as push constant shader data and, when there are none, writes the offsets to the shader data buffer. Shaders without stencil references keep the current rule.

applyShaderDataPatches and the draw cache's verify comparison also index push bytes by user data slot. For such a shader they reach only the reference word, which replay rewrites before every pass, so they are unchanged.

No console reference applies: replay emulates a missing Vulkan extension on the host. Not measured on a console.

Tested

Gentoo Linux, kernel 7.2.8, GCC 16.2.1, glibc 2.43, Release, BUILD_TESTING=ON, NVIDIA GeForce RTX 5090 (595.99.02). The branch was first tested on main 8727ddb and is now rebased onto main 4777cac; the runs below are on 4777cac unless a bullet says otherwise.

Not run on macOS or a console; on Windows only as above.

Checklist

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 6192 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6194 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6223 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6412 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6481 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7484 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7642 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7689 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 8998 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9176 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9287 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9289 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9508 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9592 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9674 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9852 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 10193 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 10442 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp

@github-actions github-actions Bot added the agc libSceAgc* label Oct 20, 2026
@ycbuzi

ycbuzi commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Windows run of 895a93e merged onto main b77bd58 (8ddb86bb40, clean), one of the hosts the description lists as not run. Windows 11 Home 10.0.26200, MinGW-w64 GCC 15.2.0 (ucrt-posix-seh r7), Release, BUILD_TESTING=ON. GPU: NVIDIA GeForce RTX 4070 SUPER, driver 616.92, picked with ANYPS5_GPU=4070 (the machine also has a GTX 1650 SUPER).

Built libSceAgcDriver and agc_driver_stencil_replay_tests with no errors or warnings; ctest --repeat until-fail:2 on agc_driver_stencil_replay and agc_driver_stencil_replay_no_gpl: each passes on both repeats, neither skipped. Not run: macOS, a console, a title.

Lindamea1 and others added 2 commits October 22, 2026 11:24
…s shader data is

A pixel shader whose stencil export is replayed gets a push constant
block that ends with the stencil reference word. When its shader data
is not in push constants (registration modules give a shader with an
image a shader data buffer, and data over the stage push range goes
there too), the block holds only that word, and buildPrepare took any
non-empty block to mean the guest buffer offsets are in push
constants: the draw threw "guest buffer offset lies outside the
shader's push constants". Only the stencilReferenceDword words before
the reference now count as shader data in push constants; with none,
the offsets go to the shader data buffer, where the shader reads them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Draw takes an optional fragment push range size. The new case sets it
to the stencil reference dword, so the shader data of a pixel shader
that reads its color with s_buffer_load_dword through a buffer
descriptor in user data is in a buffer with or without the pipeline
library, and the reference is the only push constant. The replayed
draw must color every pixel from the buffer. Without the buildPrepare
fix before it, both stencil replay registrations refuse that draw with
"guest buffer offset lies outside the shader's push constants".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@blawesom

Copy link
Copy Markdown
Contributor

Linux AMD run of the same paired protocol, on the driver family none of the runs above covered (RADV):

Machine: CachyOS, kernel 7.2.10, GCC 16.2.1, Ninja, Release, BUILD_TESTING=ON, AMD Radeon RX 9070 XT, RADV (Mesa 26.2.4, GFX1201). This branch is head 15803c7f1a, clean against main f075f91737.

Paired run — main and this branch, ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, ctest -V, the same focused set of 20 from the description:

  • 20/20 pass on both, identical per-test results.
  • Neither log contains was not found but was requested by env var VK_INSTANCE_LAYERS; both report Graphics validation tests passed, and both have no VUID- line. RADV printed no layer messages at all here (the ten Undefined-Value-ShaderOutputNotConsumed warnings of the NVIDIA run are driver-specific; the count is the same on main and this branch — zero).
  • Old code, new test: with ShaderResources.cpp from main and this branch's StencilReplay.cpp, both agc_driver_stencil_replay and agc_driver_stencil_replay_no_gpl fail with AGC graphics: guest buffer offset lies outside the shader's push constants, the exact error the description predicts — so the gap this closes reproduces on RADV, with and without the pipeline library.

One RADV difference worth noting: agc_driver_stencil_ref_export runs here rather than skipping (RADV exposes VK_EXT_shader_stencil_export), so this run covers both the native stencil-export path and the forced-replay paths on a second vendor. The lsfg-vk implicit-layer messages in the logs are from this machine's user session, appear identically in both runs, and do not come from the requested KHRONOS layer.

AI-assisted: yes (OpenCode); both runs and the old-code cross-check were executed on this machine.

@akkadbakkad89

Copy link
Copy Markdown
Contributor

In-title datapoint, PPSA20397 (Assassin's Creed Shadows), RTX 3070 + Windows 11.

Stacked 408a656e + 15803c7f on the open stencil chain (#9317 92ebac24 + #9393 + #9904 + #9905 + #9906), rebuilt the full .prx set, ran the title to the stencil draws (VK_LAYER_KHRONOS_validation active, shader cache off).

In the run before this PR was added, shader 0x407321e400 (unbounded 256-reference export) failed with AGC graphics: guest buffer offset lies outside the shader's push constants — the exact signature this fixes. With the PR in, a later run reached shader 0x407325e400's replay and proceeded to a different, later gate (stencil replay refuses sampled depth-surface feedback). Caveat: 0x407321e400 did not reappear in that run — ACS's draw order isn't fully deterministic — so this is "no longer observed + next gate reached", not a definitive in-title pass of the fixed path. No regressions: prior gates (recompile, replay-safety) still cleared by other shaders.

@Lindamea1

Copy link
Copy Markdown
Contributor Author

Thanks @blawesom and @akkadbakkad89, I've added both runs to the description. The RADV run is under Tested and GPUs tested. Assassin's Creed Shadows (PPSA20397, #8674) is now among the titles that reach this error with changes not yet on main, with your caveat about 0x407321e400.

Monster Hunter Wilds (PPSA24191), where I first found this, now reaches it in the title with open pull requests only, no local changes. I ran it on main 1e20650e8 with #10304, #9317 and #9317's four follow-ups (#9904, #9393, #9905, #9906), on the RTX 5090 with no network; details in my comment on #9317.

  • Without this PR, the title's first replayed draw of an unbounded stencil export throws AGC graphics: guest buffer offset lies outside the shader's push constants (1 of 2 runs). The other run stopped before that draw, on a multisample depth check.
  • With this PR (15803c7f1a) merged, no run had the push-constant error. In 2 of 5 runs, that draw got past buildPrepare, and the run then stopped at a later stencil replay check, stencil replay refuses depth or stencil feedback reads. The other 3 stopped before that draw, on the same multisample depth check.

That is in the description now as well.

AI-assisted: yes (Claude Code). The MH Wilds runs are from this machine.

@oneandonlydean
oneandonlydean merged commit 1eb6694 into boykopovar:main Oct 26, 2026
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants